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EXAMINER'S ANSWER 



This is in response to the appeal brief filed 8/6/08 appealing from the Office action mailed 
9/27/07. 
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(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial proceedings 
which will directly affect or be directly affected by or have a bearing on the Board's decision in 
the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection contained in 
the brief is correct. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is correct. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(8) Evidence Relied Upon 

6,757,362 Cooper et al. 3-2002 

2002/0029203 Pelland et al. 3-2002 

(Alex Millie et al., "Driver-Friendly Assistance System Interface," [online], March 14, 2002, 
[retrieved prior to August 6, 2003 ], retrieved from the Internet: 
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/shunk.standord.edu:88/Get/File- 2966/ToyotaWinterDesignReview2002Rev2[ 1 ].ppt> (21 
pages). 

(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 

Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United Stales only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

Claims 1, 3-6, 8, 10-11, 15, 17-19, 21, 23-24, 28, 30-32, 34, 36, 40, 42-46, 48-49, 53-56, 
58, and 60, are rejected under 35 U.S.C. 102(e) as being anticipated by Cooper et al. (hereinafter 
Cooper) (U.S 6,757,362 filed on Mar. 6, 2000 , and issued on Jun. 29, 2004). 

As per claims 1,15, 28, 40, and 53, Cooper teaches: 

receiving an utterance of the user (col. 2, line 39); 

obtaining utterance parameters from the utterance, the utterance parameters indicating the 
state of the user (col. 43, lines 62-67, inherent in the process of determining the user's emotional 

state based on words of the received utterance, because in order to classify the utterance into an 
emotional state type, the used method has to discriminate between the parameters of a the 
utterance. Some parameters do have in classification because they represent an emotional state 
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that could be applied to the user; and some parameters of the utterance could be neutral. They 
don't refer to any emotional state); 

determining the state of the user based upon the utterance parameters (col. 43, lines 62- 
67, wherein the Virtual Assistant (VA) determines the emotional state of the user based on the 
received utterances from the user) ; 

adjusting the voice prompt by adjusting at least one of a tone of voice of the voice 
prompt, a content of the voice prompt, a prosody of the voice prompt, and a gender of the voice 
prompt based upon the determined state of the user (col. 44, lines 1-4); and wherein obtaining 
parameters comprises: 

partitioning the utterance into segments (col. 2, lines 58-64, wherein the system identifies 
the boundaries between the spoken words, by detecting a user's pauses when recording a 
message); and 

assigning one of a plurality of classifications to each segment, each classification 
corresponding to at least one of a pliirality of states of the user (col. 2, lines 58-64, and col. 
43wherein the system assigns an emotional state, i.e. calm or angry, based on a segment or word 
choice, such as "sorry"). 

As per claim 3 and 54, Cooper teaches generating an utterance parameter vector based 
upon the utterance parameters; converting the utterance parameter vector to an indication 
representing the state of the user; and determining the state of the user based upon the indication 
(inherent in analyzing speech utterances received fi-om a user and automatically determining the 
user's emotional state, col. 43, lines 62-67). 
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As per claim 4, 17, 24, 30, and 42, Cooper teaches determining the number of segments 
for each classification, and dividing the number of segments for each classification by a total 
number of segments in the utterance (inherent in determining speech prosody, i.e. speech rate, 
loudness or volume (col. 43, line 64-65). 

As per claim 5, 18, 31, 43, and 55, Cooper teaches applying a function to the utterance 
parameter vector to generate one of a scalar, a vector of fuzzy classes, and an index representing 
the state of the user (inherently disclosed in storing information about the user's emotional state, 
for a future use, col. 44, lines 4-6). 

As per claim 6, 19, 32, 44, and 56, Cooper teaches determining that the user is in a first 
state if the scalar is greater than a predetermined threshold and that the user is in a second state if 
the scalar is not greater than the predetermined threshold (inherent in classifying the emotional 
state of the user as calm or angry, col. 43, line 67). 

As per claim 8, 21, 34, 46, and 58, Cooper teaches adjusting the content of the voice 
prompt to use content that is consistent with the determined state of the user (col. 44, lines 1-6). 

As per claims 10, 11, 23, 36, 48, and 60, Cooper teaches adjusting the prosody of the 
voice prompt, by pausing the voice prompt, to use prosody that is consistent with the determined 
state of the user (col. 47, lines 38-40, wherein the VA pauses and allows to the user to "barge 
in"). 

As per claims 45 and 49, Cooper teaches speech synthesizer module (text to speech 
engine, col. 5, line 54), and a speech storage module (. . .the VA stores temporary speech and log 
files, col. 18, lines 63-65) generates the audio waveform of the voice prompt to have a tone that 
is consistent with the determined state of the user. 
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Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the im ention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the stibject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 7, 9, 20, 22, 33, 35, 47, 57, and 59, are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Cooper in view of Pelland et al. (hereinafter, Pelland) (U.S 2002/0029203, 
published on Mar. 7, 2002). 

Cooper teaches determining the emotional state of the user (col. 43, lines 62-67). 

Cooper does not explicitly teach adjusting the tone and the gender of the voice prompt to 
use a tone that is consistent with the determined state of the user. 

Pelland in the same field of endeavor teaches adjusting the tone ([0056] and [0057], 
wherein the personal assistant (PA) adjust the voice tone to present to the user a personality that 
has a lower level of formality and higher level of humility), and the gender ([0025], wherein the 
role of the PA changes gender, influenced by the culture, and [0058], wherein the PA can switch 
between voices, by selecting a different voice file and perhaps personality rules as well) of the 
voice prompt to use a tone that is consistent with the determined state of the user. 

Therefore, it would have been obvious to a person of ordinary skill in the art at the time 
of the invention was made to modify the interactive system of Cooper by incorporating the tone, 
and gender adjusters of Pelland's, to adjust the tone voice prompt based on the emotional state of 
the user. Pelland suggests that this would provide effective ways to coordinate and handle the 
electronic communications that such devices make possible. 
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Claims 12-14, 25-27, 37-39, 50-52, and 61, are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Cooper in view of Millie et al. (hereinafter, Millie) (Admitted prior art 
"Driver-Friendly Assistance System Interface"). 

Cooper teaches all the limitations of claim 1, upon which claims 12 and 13 depend. 

Cooper does not explicitly teach wherein the system is an on-board computer used in an 
automobile or a navigation system used in an automobile; receiving information on a driving 
condition from the on-board computer or the navigation system and determining the state of the 
user based upon the information on driving condition; and adjusting a graphical character display 
corresponding to the voice prompt based upon the determined state of the user. 

Millie in the same field of endeavor teaches wherein the system is an on-board computer 
used in an automobile or a navigation system used in an automobile, and receiving information 
on a driving condition from the on-board computer or the navigation system and determining the 
state of the user based upon the information on driving condition (Page 2, wherein Chris (virtual 
passenger) determines that Pat (driver) is annoyed at unusually heavy traffic, he expresses 
sympathy about the fraflfic situation, and when he senses that the fraffic begins to clear up, he 
imitates a conversation); and adjusting a graphical character display corresponding to the voice 
prompt based upon the determined state of the user (page 1 1 , wherein the system takes pictures, 
link them the sound files, and display them on a touch-screen to further assist the driver). 

Therefore, it would have been obvious to a person of ordinary skill in the art at the time 
of the invention was made to combine Millie's feature with the system of Cooper, because Millie 
suggests that this would provide a provide a friendly system that determines the condition of the 
driver and responds with feedback to alter conditions such as loneliness and boredom (page 4). 



Application/Control Number: 1 0/66 1,152 Page 8 

Art Unit: 2626 

(10) Response to Argument 

As per independent claims 1, 15, 28, 40, and 53, applicant argues that Cooper does not 
disclose or even suggest obtaining utterance parameters that indicate the state of the user by 
portioning the utterance into segments and assigning one of a plurality of classifications to each 
segment, where each classification corresponds to at least one of a plurality of the states of the 
user. Furthermore, applicant states that the examiner does not offer any rationale, evidence, and 
basis in fact, or technical reasoning to reasonably support that, obtaining utterance parameters 
indicating the state of the user by partitioning the utterance into segments and assigning one of a 
plurality of classifications to each segment is inherent in the disclosure of Cooper (Appeal Brief, 
pages 16-17). 

The examiner notes that Cooper determines the user's emotional state based on user's 
information such as spoken words, as admitted by applicant (Appeal Brief, page 16-17), and as 
stated in Cooper's (col. 43, line 52 - col. 44, line 6). Additionally, Cooper discloses a database 
storing words or phrases that are classified. In another word, each word, phrase, or sentence 
stored in Cooper's database corresponds to a classification fi-om a plurality of classifications. 
When the virtual system determines that the user is angry, the interactive mode of the virtual 
assistant access it's database and selects the set of classified words, i.e. "submissive words" such 
as "sorry", "regret", and 'apologize". When the virtual assistant feels a polite discourse from the 
user, it uses words , phrases or sentences such as "please", "thank you", "excuse", "pardon", 
"may I", and "would you mind" that are classified as polite discourse. Therefore, Cooper 
teaches assigning one of a pliirality of classifications to each segment of spoken text by the user, 
no matter what the concept of "the segment" is, i.e. a word, phrase, or sentence. 
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Furthermore, Cooper's virtual system adapts its behavior based on the emotional state of 
the user, whether the user uses words associated with polite discourse (col. 2, lines 46-64) or 
whether the user is calm or angry (col. 43, line 52 - col. 44, line 6). Hence, obtaining utterance 
parameters indicating the state of the user is inherently disclosed in the teaching of Cooper. 
Otherwise the virtual system would not be able to know the emotional state of the user in order 
to adapt its behavior accordingly. Still, in order to classify the utterance into an emotional state 
type, the used method has to discriminate between the parameters of the utterance. Some 
parameters do have a weight in the utterance classification because they represent an emotional 
state that could be applied to the user; and some parameters of the utterance have less or no 
weight in the utterance classification, they are neutral. They don't represent or refer to any 
emotional state. 

Applicant states that Cooper merely discloses that the virtual assistant could determine 

the user's emotional state (calm or angry) based on user's information such as the user's voice 
volume, word choice, and speed rate, but does not disclose at all how the user's emotional state 
can be determined based on such information. The examiner notes that claims 1, 15, 28, 40, and 
53 do not claim any details how the user's emotional state can be determined based on such 
information. All what is claimed prior to assigning one of a plurality of classifications to each 
segment, where each classification corresponds to at least one of a plurality of the states of the 
user, is receiving an utterance, and partitioning the utterance into segments, which is known 
standard in the speech recognition process. 

Applicant argues that since Cooper fails to disclose that each segment of the utterance is 
assigned a classification indicating one of plurality of states, he does not teach the limitation of 
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"generating an utterance parameter vector based upon the utterance parameters and converting 
the utterance parameter vector to an indication representing the state of the user" (Appeal Brief, 
page 18). The examiner notes that Cooper does teach assigning each segment of the utterance a 
classification indicating one of plurality of states, as explained above (see col. 2, lines 58-64, and 
col. 43, lines 52-67, wherein the virtual assistant determines the emotional state of the user based 
on user's spoken words, given that "the segment" is broad concept interpretable as a word, 
phrase, sentence, or paragraph). Therefore, "generating an utterance parameter vector based 
upon the utterance parameters and converting the utterance parameter vector to an indication 
representing the state of the user" is inherently disclosed within the teaching of Cooper. After an 
input utterance is received by the virtual assistant system of Cooper, it gets segmented into 
frames for speech recognition process (col. 6, lines 32-37). A frame could be a word, phrase, or 
sentence. Each frame is approximated by one of a fixed set of possible audio vectors. Those 
utterances segments are obviously compared with training data. The training data is stored 
within the database in the form of vectors, wherein each vector refers to a different emotional 
state; and therefrom determining the emotion state that best matches the data segment, and 
identifying the emotional state of the user. 

As per claim 3 and 54, applicant argues that there is no rationale, evidence, basis in fact, 

or technical reasoning as to why it is inherent in Cooper that the utterance parameter vector is 
generated as in claims 3 and 54, especially when Cooper does not even mention the term 
"vector" anywhere (Appeal Brief, pages 19-20). There is no other way other then generating 
vectors based on the utterance parameters to determine the emotional states of the user. 
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especially when Cooper teaches that the emotional state of the user is based on words spoken by 
the user (col. 43, lines 62-65). 

What has been done by Cooper to determine the emotional state of the user is standard in 
the field of speech recognition. An utterance is received; the utterance is segmented into 
segments; each segment is represented by one of a fixed set of possible vector audio parameters, 
wherein each parameter vector component varies with each different emotional state; and as a 
result, the emotional state of the user is determined. Therefore, generating an utterance 
parameter vector based upon the utterance parameters is inherently disclosed with the teaching of 
Cooper. 

Regarding mentioning the term "vector", the examiner notes that even though Cooper 
does not mention the term "vector". Cooper provides the same fiinction bases on the same type 
of data, which is determining the emotional state of a user based on words spoken by the user, 
and using parameters vectors is an inherent step within the process of speech analysis and 
classification. Therefore, the parameters vectors are inherently used, as explained above, and do 
not have to be explicitly mentioned in Cooper's. 

As per 4, 17, 30, and 42, applicant argues that the examiner does not explain why the 
utterance parameter vector must necessarily be generated in Cooper as recited in above 
mentioned claims rather by some other vector generation method. (Appeal Brief, page 23). The 
examiner notes that since both, applicant and Cooper rely on determining of the emotional state 
of the user based on the same type of data, utterances spoken by the user; and in order to analyze 
and classify speech, it has to be segmented into segments (as in Fig. 37), and parameters 
indicating the state of the user have to be generated therefrom, as explained above. Therefore, 
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Cooper's method of determining vector's probability must be based on the total number of 
segments within the spoken utterance and the number of segments for each classification, which 
is a standard method of determining the probability or the likelihood of a word type occurrence. 

As per the rest of the claims and combinations of prior art reference (Cooper in view of 
Pelland, and Cooper in view of Millie), applicant has no further arguments beside the ones 
mentioned above. Therefore, all the combinations of prior art reference mentioned above are 
valid, and all other dependent claims are rejected for the same reasons as set above. 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the Related 
Appeals and Interferences section of this examiner's answer. 



For the above reasons, it is believed that the rejections should be sustained. 

RespectfiiUy submitted, 

/Abdelali Serrou/ 
Examiner, Art Unit 2626 

Conferees: 

Patrick Edouard (SPE: AU 2626)/Patrick N. Edouard/ 
Supervisory Patent Examiner, Art Unit 2626 

/Talivaldis Ivars Smits/ 

Talivaldis Ivars Smits (Primary Examiner: 2626) 



/Abdelali Serrou/ 

Abdelali Serrou (Examiner: 2626) 

10/29/2008 



